Skip to content

Fail loudly instead of skipping cea_module fixture on broken CI builds - #151

Open
djkees wants to merge 1 commit into
nasa:mainfrom
djkees:up/pytest-cea-fixture-skip-masks-failures
Open

Fail loudly instead of skipping cea_module fixture on broken CI builds#151
djkees wants to merge 1 commit into
nasa:mainfrom
djkees:up/pytest-cea-fixture-skip-masks-failures

Conversation

@djkees

@djkees djkees commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

The session-scoped cea_module pytest fixture in source/bind/python/tests/conftest.py, which nearly every Python test depends on, called pytest.skip(...) when import cea or cea.is_initialized() failed. A broken build (compile failure, missing extension, botched CMake step) made every dependent test report "skipped" rather than "failed," and pytest exits 0 on an all-skipped run — so basic_build.yml's python-tests job could show green even though nothing actually ran.

djkees#89

Changes

  • conftest.py: the fixture now checks the CI/GITHUB_ACTIONS environment variables (set automatically by GitHub Actions runners). When set, an import or init failure calls pytest.fail(...) instead of pytest.skip(...), so the run errors out with a nonzero exit code. Outside CI (e.g. local dev without a built extension), the original skip behavior is unchanged.

Testing

  • pytest source/bind/python/tests -q (real dev environment, extension built): 116 passed, confirming no regression.
  • Manual before/after check with a throwaway test that poisons sys.modules["cea"] to force an import failure: without CI set, 1 skipped (unchanged); with CI=true, 1 error / Failed: cea import failed: ... (new behavior).

Compatibility / Numerical behavior

  • No expected changes to numerical results

Test-infrastructure only; no solver/binding logic touched.


Drafted with Claude's assistance

  • Verified by reading the original conftest.py and reproducing both the old (skip) and new (fail) behavior directly, in both a clean sandbox and a real dev environment, before and after the change.
  • Full existing Python test suite (116 tests) re-run and confirmed passing after the change.

#208)

In CI, import/init failure in the fixture now calls pytest.fail instead
of pytest.skip, so a broken build can't report as a green, all-skipped run.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant